Relation extraction using distant supervision: a survey

Relation extraction is a subtask of information extraction where semantic relationships are extracted from natural language text and then classified. In essence, it allows us to acquire structured knowledge from unstructured text. In this article, we present a survey of relation extraction methods that leverage pre-existing structured or semi- structured data to guide the extraction process. We introduce a taxonomy of existing methods and describe distant supervision approaches in detail. We describe, in addition, the evaluation methodologies and the datasets commonly used for quality assessment. Finally, we give a high-level outlook on the field, highlighting open problems as well as the most promising research directions

Toloka Visual Question Answering Benchmark

In this paper, we present Toloka Visual Question Answering, a new crowdsourced dataset allowing comparing performance of machine learning systems against human level of expertise in the grounding visual question answering task. In this task, given an image and a textual question, one has to draw the bounding box around the object correctly responding to that question. Every image-question pair contains the response, with only one correct response per image. Our dataset contains 45,199 pairs of images and questions in English, provided with ground truth bounding boxes, split into train and two test subsets. Besides describing the dataset and releasing it under a CC BY license, we conducted a series of experiments on open source zero-shot baseline models and organized a multi-phase competition at WSDM Cup that attracted 48 participants worldwide. However, by the time of paper submission, no machine learning model outperformed the non-expert crowdsourcing baseline according to the intersection over union evaluation score.Comment: 16 pages; see https://toloka.ai/challenges/wsdm2023/ for more detail

APCNN: Tackling Cclass imbalance in relation extraction through aggregated piecewise convolutional neural networks

One of the major difficulties in applying distant supervision to relation extraction is class imbalance, as the distribution of relations appearing in text is heavily skewed. This is particularly damaging for the multi-instance variant of relation extraction. In this work, we introduce a new model called Aggregated Piecewise Convolutional Neural Networks, or APCNN, to address this problem. APCNN relies on the combination of two neural networks, a novel objective function as well as oversampling techniques to tackle class imbalance. We empirically compare APCNN to state-of-the-art approaches and show that it outperforms previous multi-instance approaches on two standard datasets

Scalpel-cd: leveraging crowdsourcing and deep probabilistic modeling for debugging noisy training data

This paper presents Scalpel-CD, a first-of-its-kind system that leverages both human and machine intelligence to debug noisy labels from the training data of machine learning systems. Our system identifies potentially wrong labels using a deep probabilistic model, which is able to infer the latent class of a high-dimensional data instance by exploiting data distributions in the underlying latent feature space. To minimize crowd efforts, it employs a data sampler which selects data instances that would benefit the most from being inspected by the crowd. The manually verified labels are then propagated to similar data instances in the original training data by exploiting the underlying data structure, thus scaling out the contribution from the crowd. Scalpel-CD is designed with a set of algorithmic solutions to automatically search for the optimal configurations for different types of training data, in terms of the underlying data structure, noise ratio, and noise types (random vs. structural). In a real deployment on multiple machine learning tasks, we demonstrate that Scalpel-CD is able to improve label quality by 12.9% with only 2.8% instances inspected by the crowd

GIANT: Scalable Creation of a Web-scale Ontology

Understanding what online users may pay attention to is key to content recommendation and search services. These services will benefit from a highly structured and web-scale ontology of entities, concepts, events, topics and categories. While existing knowledge bases and taxonomies embody a large volume of entities and categories, we argue that they fail to discover properly grained concepts, events and topics in the language style of online population. Neither is a logically structured ontology maintained among these notions. In this paper, we present GIANT, a mechanism to construct a user-centered, web-scale, structured ontology, containing a large number of natural language phrases conforming to user attentions at various granularities, mined from a vast volume of web documents and search click graphs. Various types of edges are also constructed to maintain a hierarchy in the ontology. We present our graph-neural-network-based techniques used in GIANT, and evaluate the proposed methods as compared to a variety of baselines. GIANT has produced the Attention Ontology, which has been deployed in various Tencent applications involving over a billion users. Online A/B testing performed on Tencent QQ Browser shows that Attention Ontology can significantly improve click-through rates in news recommendation.Comment: Accepted as full paper by SIGMOD 202

Nessy: A Neuro-Symbolic System for Label Noise Reduction

Noisy labels represent one of the key issues in supervised machine learning. Existing work for label noise reduction mainly takes a probabilistic approach that infers true labels from data distributions in low-level feature spaces. Such an approach is not only limited by its capability to learn high-quality data representations, but also by the low predictive power of data distributions in inferring true classes. To address those problems, we introduce Nessy, a neuro-symbolic system that integrates deep probabilistic modeling and symbolic knowledge for label noise reduction. Our deep probabilistic model infers the true classes of data instances with noisy labels by exploiting data distributions in an underlying latent feature representation space. For data instances where inference is not reliable enough, Nessy extracts symbolic rules and ranks them according to several utility metrics. Top-ranking rules are injected into the deep probabilistic model via expectation regularization, i.e., via a posterior regularization term constraining the class distribution in the objective function. In a real deployment over multiple relation extraction tasks, we demonstrate that Nessy is able to significantly improve the state of the art, by 7% accuracy and 10.7% AUC on average.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Web Information System

Catalytic Antibodies in Bipolar Disorder: Serum IgGs Hydrolyze Myelin Basic Protein

The pathogenesis of bipolar affective disorder is associated with immunological imbalances, a general pro-inflammatory status, neuroinflammation, and impaired white matter integrity. Myelin basic protein (MBP) is one of the major proteins in the myelin sheath of brain oligodendrocytes. For the first time, we have shown that IgGs isolated from sera of bipolar patients can effectively hydrolyze human myelin basic protein (MBP), unlike other test proteins. Several stringent criteria were applied to assign the studied activity to serum IgG. The level of MBP-hydrolyzing activity of IgG from patients with bipolar disorder was statistically significantly 1.6-folds higher than that of healthy individuals. This article presents a detailed characterization of the catalytic properties of MBP-hydrolyzing antibodies in bipolar disorder, including the substrate specificity, inhibitory analysis, pH dependence of hydrolysis, and kinetic parameters of IgG-dependent MBP hydrolysis, providing the heterogeneity of polyclonal MBP-hydrolyzing IgGs and their difference from canonical proteases. The ability of serum IgG to hydrolyze MBP in bipolar disorder may become an additional link between the processes of myelin damage and inflammation

Oxidative Stress-Related Mechanisms in Schizophrenia Pathogenesis and New Treatment Perspectives

Schizophrenia is recognized to be a highly heterogeneous disease at various levels, from genetics to clinical manifestations and treatment sensitivity. This heterogeneity is also reflected in the variety of oxidative stress-related mechanisms contributing to the phenotypic realization and manifestation of schizophrenia. At the molecular level, these mechanisms are supposed to include genetic causes that increase the susceptibility of individuals to oxidative stress and lead to gene expression dysregulation caused by abnormal regulation of redox-sensitive transcriptional factors, noncoding RNAs, and epigenetic mechanisms favored by environmental insults. These changes form the basis of the prooxidant state and lead to altered redox signaling related to glutathione deficiency and impaired expression and function of redox-sensitive transcriptional factors (Nrf2, NF-κB, FoxO, etc.). At the cellular level, these changes lead to mitochondrial dysfunction and metabolic abnormalities that contribute to aberrant neuronal development, abnormal myelination, neurotransmitter anomalies, and dysfunction of parvalbumin-positive interneurons. Immune dysfunction also contributes to redox imbalance. At the whole-organism level, all these mechanisms ultimately contribute to the manifestation and development of schizophrenia. In this review, we consider oxidative stress-related mechanisms and new treatment perspectives associated with the correction of redox imbalance in schizophrenia. We suggest that not only antioxidants but also redox-regulated transcription factor-targeting drugs (including Nrf2 and FoxO activators or NF-κB inhibitors) have great promise in schizophrenia. But it is necessary to develop the stratification criteria of schizophrenia patients based on oxidative stress-related markers for the administration of redox-correcting treatment

A conserved cysteine cluster, essential for transcriptional activity, mediates homodimerization of human metal-responsive transcription factor-1 (MTF-1)

Metal-responsive transcription factor-1 (MTF-1) is a zinc finger protein that activates transcription in response to heavy metals such as Zn(II), Cd(II) and Cu(I) and is also involved in the response to hypoxia and oxidative stress. MTF-1 recognizes a specific DNA sequence motif termed the metal response element (MRE), located in the promoter/enhancer region of its target genes. The functional domains of MTF-1 include, besides the DNA-binding and activation domains and signals for subcellular localization (NLS and NES), a cysteine cluster (632)CQCQCAC(638) located near the C-terminus. Here we show that this cysteine cluster mediates homodimerization of human MTF-1, and that dimer formation in vivo is important for basal and especially metal-induced transcriptional activity. Neither nuclear translocation nor DNA binding is impaired in a mutant protein in which these cysteines are replaced by alanines. Although zinc supplementation induces MTF-1 dependent transcription it does not per se enhance dimerization, implying that actual zinc sensing is mediated by another domain. By contrast copper, which on its own activates MTF-1 only weakly in the cell lines tested, stabilizes the dimer by inducing intermolecular disulfide bond formation and synergizes with zinc to boost MTF-1 dependent transcription